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To learn more about the specific nature of the 
teaching and learning problems involved, researchers conducted a 
clinical study of 20 high school students enrolled a BASIC course. 
Investigators presented each student with a sequence of eight 
programming problems, ranging from easy to difficult. They asked 
questions to track student thinking and intervened in student 
difficulties with graduated levels of assistance. A coding system was 
used to record the type of difficulty students encountered, the 
amount of help needed, and the correctness of solutions. 
Experimenters noted whether errors were omissions of a necessary 
element, inappropriate migrations of an element trom one command to 
another, errors in sequencing the elements, or other mistakes. Data 
analysis provided information about loci of difficulty in three 
aspects of programming behavior: attitudes, knowledge base, and 
problem-solving strategies. Inadequacies in students* knowledge base 
about BASIC were revealed, with most errors occurring at the level of 
application. Findings suggest that programming instruction might 
place greater emphasis on encouraging students to prompt themselves 
with strategic questions about problematic situations, on helping 
them achieve more consolidated knowledge of the details of computer 
languages, and on addressing attitudinal and confidence faccors in 
programming. (32 references) (Author/MES) 
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Loci of Difficulty in Learning to Progru 



The evidence continues to accumulate that students in priaary and seconftary 
school studying progrsMiing do not learn to program very well* Studies of 
Logo prograHiing conducted at Bank Street showed that students eaerged trom 
instruction of a nondirective sort with very liaited Mstery of the basics of 
Logo (Pea ft Kurlandt 1984at 1984b) « Linn (1985) » reporting on an extensive 
sicxvey of BASIC instruction, noted that student learning was quite United at 
aost sites t although better results appeared at a few sites with higher 
ability students and especially mindful instruction. In a recently coapleted 
stndy, Bank Street researchers exaained the cpapetence of hi|^ school 
students involved in a two year conputer science sequence involving 
considerable prograMing instruction as well as related aspects of coaputer 
use (Kurlandt Pea, Cleaentt * Nawbyt in press). At the end of the 
instruction, aany students still displayed surprising shortfalls in their 
coapetence. 

To be sure, education aspires to greater achleveaent in virtually every 
area of the curriculua. Still, the gap between achleveaent and aspiration 
seeas onusvially large In the case of prograaaing. Students caerge froa aany 
straightforwsrdly and seeaingly coapetently conducted prograas of instruction 
with anch acre Halted aastery than one would have expected. The 
circuastances pose a puzxle both for psychology and pedagogy: How to explain 
the difficulties of prograaaing that stand in the way of student progress? 

One possible answer turns to the deaands prograaaing aakes on 
problea-solving ability. Prograaaing is a problea-solving Intensive subject. 
Mhile students aay asnage well enough in aany school subjects by learning the 
facts and rote procedures and delivering thea back in hoaework and quizzes, 
prograMing instruction routinely asks students to create their own prograas 
to do a variety of tasks. Accordingly, prograaaing routinely presses 
students for a variety of analytical and coapositional coapetencies that 
other subjects, as noraally taught, do not deaand. Paradoxically, aost 
prograaaing instruction does little to teach such coapetencies directly; 
rarely does prograaaing instruction explicitly address the sorts of 
heuristics or aanagerial strategies that aigfat abet the problea-solving 
process. There is soae evidence to suggest that when the instruction 
includes explicit attention to such natters, students do better (Cleaents, 
1988; Cleaents ft Gullo, 1984). Outside the area of prograaaing, there is 
evidence to suggest the explicit teaching of heuristics and aanagerial 
strategies can iaprove problea solving substantially (Schoenfeld, 1982; 
Schoenfeld ft Hermann, 1982). 

One ai^t also seek an understanding of the difficulties of progrsaaing 
in a second direction: As well as a problea-solving intensive subject, 
progra»iing is a precision- intensive subject. An able prograaaer needs to 
know the key coaaands, understand the purposes that they serve, grasp what 
actions they effect in the world of the coaputer, and respect the syntactic 
requireasnts of their proper use. To be sure, such knowledge is aerely the 
low- level ^'database*' that supplies inforaation for the higher order problea 
solving students of prograsalng need to do. However, it is by no aeans clear 
that students have that **database" well in hand. One conspicuous shortfall 
is the difficulties students coaaonly display in hand-executing given 
prograas or prograas they have already written (Pea ft Kurland, 1984a, 1984b; 
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perkimi, Haacock, Hobb•^ Martin, ft and Sinaons, in presa). Hand-execution 
requirea no problea aolvlng; it la a purely algoritladc task calling only for 
a praciae graap of tjhat actions the various coaaands effect* Yet students 
routinely have difficulty with tasks of this sort, demonstrating that the 
databaaa of coaaanda and their effects in the world of the coaputer needs 
extenaion and repair. 

Perhapa it ia soaeidiat surprising to think that the low-level knowledge 
of coaaanda aay be a aajor aource of atudents* difficulties. But one does 
well to reaeaber that the circuaatancea of prograaaing call fpr an 
exceedingly high accuracy rate. If one recalla with accuracy 90% of the 
datea on a hlatory quiz and faila to retrieve or retrievea erroneoualy the 
reatt one haa turned in a creditable perforaance. If one handlea with 
accuracy only 90% of the coaaanda in a particular prograa, alaoat certainly 
the prograa will fall. A coaputer prograa ia a highly interdependent 
atrttctnre» idiere errors in one locale have effecta that propagate to ^il 
the whole perforaance of the prograa. Consequently, although the total 
aaount of rote inf oraation a atudent needs to aaater for proKraaalng aay be 
aubatantially leaa than that a atudent needa to aaater for history or a 
foreign language , the fullneas of aastery the student requires to do well is 
auch higher* 

In suaaary, prior research suggests that prograaaing aay prove a 
difficult aubject for atudents both because of its problea-aolving intenaive 
character and ita precialon-intenaive diaracter. . But auch h propoaal deaands 
explication. What aorta of problea-aolving atrategiea aight be needed? If 
atudents have trouble aaateking coaaanda, what foraa doea auch trouble take? 
Can it be aaid that either problea-aolving akills or shortfalls ir knowledge 
base lie at the heart of the aatter, difficultiea with the one priaarily 
reflecting difficultiea with the other? 

With such questions aa these in aind, we undertook a clinical study of 
hish achool students in the second seaeater of a BASIC courae. We sat with 
students as they atteapted certain prograaaing probleas and helped then in 
systeaatic ways when they encountered difficulties. The students* 
interactions with the coaputer and our interactions with the students were 
consolidated into protocols and subjected to several sorts of analyses. The 
results illuainate the loci of difficulty in learning to prograa. 



Method 

SubJecta 

Tlienty hi|^ achool atudenta participated in the atudy. Ranging frca 10th to 
12th gradera» they conaiated of 11 girla 4ind 9 boya. The atudenta were 
enrolled in the aecond aeaeater of an elective firat-i^ear BASIC course. They 
were all taught by the aaae teacher, irtio had a profeaaional level of aaatery 
of prograaaing. Bach atudent participated for one 45-ainute aeasion. Itie 
instruction in the BASIC courae seeaed clear, solid, and atraightforward, 
with conalderable practice tiae and individual attention. Preaentation of 
new coaasnda and concepU proceeded in a very structured way, with ayateaatic 
explanation and workbook exerciaea. Ihe priaary atudent activity was working 
throu^ sets of prograaaing probleaa; the probleaa tended toward buaineas 
natters such as record keeping or processing a data file. Tomrd the end of 
the aecond seaeater, the atudents were studying two-diaensional arrays. 
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Programing tasks 

In the experimental sessions » the Investigator showed each subject a series 
of eight programing probleas. The problaas were sequenced from relatively 
easy to fairly difficult, each new problea building on the one before. The 
sequence of eight was based on the POR-NBXT loopt asking the students to 
write prograas that would produce various patterns of stars (asterisks) on 
the screen. Although the students had been exposed In class to all the 
concepts and prlaltlves needed to write the prograas^ they aay not have 
encountered thea In the context of producing a graphic design. 

To ccmvey the character of the problea sequence, we describe several of 
tlie problsM. Problea 1 asked for a prograa that would produce a coluan of 
10 stars; problea 3 called for a prograa that would aak the user for an liqiut 
(a nuaber) and then print a coluan of that aany stars. Problea 4 called for 
a prograa that would ask for an Input and then print that aany stars 
horlaxmtally. Problew S requires a prograa that asked for an Input and then 
printed a square of stars. With an Input of S» the prograa would produce: 

s s s s s 

s s s s s 
9 * s s s 
s s s s s 



Problea 8 called for a prograa that would print a hollow square of stars 
deteralned by the user's Input. With an Input of 5, the prograa would 
produce: 

s s « s s 
m s 
m * 
s s 
s s s r « 



Procedure 

Each session was held during the student's regular BASIC class period. The 
experlaenter and student worked together In a separate rooa using a Digital 
Rainbow personal coaputer. The experlaenter explained the purpose of the 
study ~ to understand what aspects of progranaing are easy or hard to learn. 
The experlaenter explained that there were prograaalng probleas for the 
student to work on; while the student worked, the experlaenter would watch, 
take notes, and help If the student had trouble. The experlaenter also 
realnded the student that the session would be audlotaped. 

The experlaenter then presented the sequence of* probleas and asked the 
student to choose one to begin that was "not too easy, but not too hard a 
challenge." As the student worked, the experlaenter watched, recording on 
paper the atodent's code and other notes such as the student's affect during 
the session. Occasionally the experlaenter asked questions to track the 
student's thinking. When the student encountered a difficulty he or fi^^a was 
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uoable to resolve alone, the experinenter intervened by aaklne questions or 
providing inforjMtion* The experisenter worked with the student until the 
prograa ran successfully* Then the student was asked to work on the next 
problea in the sequence. This procedure was repeated until tiae ran out. 

Ifhen the student encountered a diff iculty, the experiaenter first asked 
questions desisned to provoke strategic thinking* We called this type of 
question a **proapt*'' By definition, pronpts were questions that did not 
require a knowledge of the true nature of the difficulty, questions that in 
principle students could have thought to ask theaselves. Soae typical 
pronpts were: ''MHit's the first thing you need to tell the computer to do?**; 
**What does this seaioolon do?"; **How would you describe the problea to 
yourself?** As the ezaaples suggest, aoae pronpts were general and could be 
used in any problea-solving situation, whereas others were specific to 
prograaaing* The prcapts were generated by the experiaenter according to the 
experiaenter's Judgaent of the level of specificity needed* 

If a couple of proapts did not help the student to overcoae the 
difficulty, the experiaenter offered aore specific assistance. This type of 
intervention was called a **hint*** By definition^ hiiics were questions or 
coaaents that reflected the experiaenter *8 understanding of the solution* A 
hint could be a leading question or a bit of inforaation. Hints sounded like 
this: **Can you think of a coaaand to get the coaputer to ask you for a 
nuaber?**; **your problea is to repeat soaething several tiaes» so do you know 
a coaaand for that?**; **Is there soae way you could use a seaicolon?** 

If a couple of hints did not provoke progress » the esqieriaenter provided 
an exact solution to the iaaediate dileaaa so that the student could proceed 
with the rest of the prograaaing. These were called **provides •** 
Characteristic provides were: **l(rite INPUT 'How aany stars do you want*, N**; 
**use a POR-NEXT loop**; **Put a seaicolon after the print stateaent*** The 
experiaenter atteapted to explain answers idien providing thea* Note that not 
all **provide8** directly gave the student code. Por instance, if the 
student's iaaediate problea was to retrieve the naae of a relevant coaaand, 
and proapts and hints failed, the experiaenter would provide the naae of the 
coaaand, but not details on how to use it. The experiaenter would then wait 
to see whether the student could follow through with the coaaand, before 
intervening further with proapts, hints, and provides. 

The escalations froa proapt to hint to provide not only helped the 
student but served as a probe of the student's level of aastery and 
understanding. The aore support the student required, the less the student 
could acooaplish alone. A successful proapt auBtested that the student could 
benefit froa learning to self --proapt in the saae way. At the other extreae, 
a provide preceded by several unsuccessful proapts and hints Indicated very 
Halted knowledge and understanding of the particular difficulty. 

Althou^ the experiaenter generally atteapted the progression froa 
proapts to provides, soaetiaes the experiaenter aoved directly to hints or 
provides. This occasioially happened by aistake, but aore often because the 
general perforaance of the student and the studc^st's increasing frustration 
indicated that aore direct help was needed to aaintain attention and 
involveaent. 
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DaU 

Data collacted during each seaaion included notea taken by the eacperlMnter, 
code witten by the atudent that ma tranacribed by the experiaenter, and an 
audiotape of the converaation. The audiotape waa tranacribed later and the 
notea and code interpolated to yield a verbatia protocol of the aeaaiono 



Coding ayatea 

Ibe eoqperlMntera developed a coding ayatea to apply to the protocola. The 
coding ayatea claaaif led the atudanta* auoceaa or dlfficultiea idth the atepa 
we identified aa neceaaary to coaplete the prograa in queation — the 
**critlcal ooaponenta" diacuaaed below. The atudaata' aocoeaa or dlfficultiea 
were inferred bf the aoorera froa the atudenta' verbally expreaaed ideaa and 
actiona at the keyboard that furthered the devaloiaMnt of the prograa or that 
the atudent hoped would further ita developaent. Coaaenta like "I need aoae 
kind of a loop"* or "I can uae INPUT*' would lead to one or aore acorea. Side 
reaarka like, "Thia ia a anap*" or **Thia ia tough** would not. 

The coding ayatea aought to capture three aapecta of the atudenta* 
prograaaing behavior: (1) the level of organization at idiich atudenta evinced 
a difficulty, for Inatance overall organization of a prograa veraua detaila 
of the handling of a particular coaaand; (2) how auch help a atudent needed 
with a particular atep — none* a proapt, a hint, or a provide; (3) the 
yorrectneaa of a atudent* a handling of a atep — waa it correct, waa the 
atudent unable to provide any reaponae, or what aort of error did the 
atudent 'a reaponae diaplay? 

Preaenting a aaaple interaction with a atudent wAll facilitate 
deacribing the coding ayatea. For auch a saaple, auppoae that a atudent ia 
addreaaing problea 4, iHiich called for a prograa to print a horizontal row of 
atara of length deterained by the uaer'a input. The atudent recognizee the 
need to get inforaation into the prograa, aentiona uaing READ, and then 
appeara stuck. The experiaenter aaya, **Can you think of another coaaand you 
aight uae?** The atudent aaya, **0h yeah, INPUT.** 

Critical coaoonenta of a orograa. To facilitate coding atudenta* 
reaponaea for each prograaaing problea, we identified thoae coaponenta of a 
aolution that were, we felt, critical to reaolvlng the problea, Fbr exaaple, 
we eatabliahed four critical coaponenta for problea 4: an INPUT atateaent, a 
FOR-NBXT loop with a variable containing the input nuaber for ita upper 
liaitt a PRINT followed by a atar and aeaicolon* and apacea between the 
atara» effected by a atateaent auch aa **PRINT ** *;** rather than **PRINT 
***;**• For each atudent who coapleted problea 4 or nearly coapleted it 
(interrupted by the end of the peiriod), the atudent* a perfonnnce on eadi one 
of theae critical prograa coaponenta waa acored. Accordingly, the aaaple 
epiaode above would be acored aa part of the INPUT coaponent. Siailar 
critical coaponenta were defined for all th^ probleaa and atudenta were 
ccored accordingly on any problea they coapleted or alaoat coapleted. 

Level. The perf oraance of each atudent on a given prograa coaponent was 
coded according to the atudent* a handling of four aajor prograaaing levela: 
decoapoaition, need foraulation, coaaand finding, and application. 
Decoapoaition referred -to reaJitlnc; the ne^d for the tn^ograa ufW^puutuL ~ in 
the caae of Problea 4 and INPUT* realizing that aoae way of getting 
inforaation in waa required. Need foraulation referred to foraulating in 
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sharper terao uliat the coaponent should do — in the case of Problem 4 saying 
soMthing like» need a coanand that reads an input fros the user."* 
^ — IIP^ finding ritferred to retrieving an appropriate coMsud — WPOT in the 
case ot the enaple. Application referred to following through with correct 
code using the cownand a correct INPUT line in this case. Considering 
again the saaple episode, the student's effort to retrieve INPUT is an 
exaaple of co— and finding* 

The level sche»e provided one approach to gauging the extent to irtiich 
students suffered f roe weaknesses in their BASIC "database" versus higher 
order difficulties as in decomposing the prograMing problem. The four 
levels were considered a logically necessary sequence for any program 
coaponent. All four levels were given distinct scores for each critical 
conpQontt evwA thoufh a studsnt aight not display any overt sign of passing 
throu^ a particular level. Scoring each level was possible becanse (a) a 
student always coapleted the coaponent, even if with help froa the 
experiaentert and (b) in those cases where a student gave no overt evidence 
of a levelt it wss scored by inference in light of the student's perforaance 
on a subssquent level. For exaq^lot if a student inasdiately wrote a correct 
coBBMit line — an act at the application level ~ by inference the scorers 
classified the decoaposition» need foranlation, and iijaainil finding levels as 
"spontaneously correct" (see below for the aeanings of these terns). For a 
CO'. treating exaaple, a student requiring the experiaenter's help with each 
level in turn would be scored accordingly at each level. 

Help. Students' responses were classified according to the help 
required to elicit then soontaneous. or with the help of a proapt. hint. 
or provide. This classification did not assuae that a subject's response to 
a proapt, hint, or provide was correct; by definition, reeponees to providee 
were always correct since the experiaenter directly provided the needed 
inforaation, but responses to ^roapts and hints were often incorrect. Note 
tiiat the classification of help froa the experiaenter as a proapt, hint, or 
provide was done froa the protocols, without reference to the experiaenter 's 
original intentions, which were in any case not indicated in the protocols.. 
Referring to the saaple case again, the students' retriev^'J of DRVT would be 
classified as in response to a proapt, because the experiaenter 'e question, 
"Can you think of another coaaand you alght uee?" is a <viestion not betraying 
My particular knowledge of the answer, a question a student under siailar 
circuastances eight well ask hiaself or he2*self . 

Corlrectness. Finally, the correctness of a response was clatYified into 
one <tf several categories: correct: oaissions failure to include a needed 
eleaent in a response, as for instance in oaitting a seaicolon on PRINT or 
leavii« out the FOR-NBXt altogether; aigrations cases where a synUctic 
eleaent froa one coaaand intruded into another, as for exaaple in PRINT S 
STBP 2; sequence errors where an eleaent was aispositiooed in a progrsa, 
as in placing one after another FOR-NBXT loops that should be nested; 
aistakes — a catch-all category for erroneous responses not otherwise 
classified. In the exaaple, the subject's response of INPUT would be 
classified as correct; the subject's prior response of READ would be 
classified as a aigration (of RBAD into the territory of INPUT) . The scoring 
would alao note that the student spontaneoualy rejected this aigration. 
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The coding oroce— 

The principal scorer went through each protocol , looking for retponf es and 
claaaifylng th« into a eet of 4 x 4 charts, one for each critical coaponent 
of each prohlea, with the help dlaenelon across the top and the level 
diaension dom the aide. An entry in a chart included a letter code tc 
indicate correctness and an annotation to indicate idMit response in the 
protocol the entry referred to. A second scorer scored a randonly selected 
suhsaaple of the protocols Independently of the first scorer. After the 
wdmpmer of Interjudge agreeaent was esUblished (see below), the principal 
scorer's classifications were adopted for all further analyses. 



Teacher's grades and ratings 

The experlaenters collected froa the teacher at the end of the tera the 
studnts' final oourae gradse. Ihey also collected the teacher's ratings for 
each student on a three point scale (low, asdiua, high) for three attitodinal 
factors: aotivation, enjoyamt, and persistence. The teacher was encouraged 
to take these terns in their everyday senses and was given no aore specific 
guidance about their interpretation. Both the grades and the attitude 
ratings inevitably raise issues about inter judge agreeaent, since there wu 
no second grader or rater for a crosscheck. However, we felt that the 
teacher's professional role and year-long faailiarity with the students put 
her in a good position to assign objective grades and ratings. Also, as the 
analysis will reveal, substantial relationships eaerged between the teacher's 
grades and ratings and the data gathered by the experlaenters, foraing a 
coherent picture that argues indirectly for the soundness of the grades and 
ratings. 



Results 

Coding reliability 

Occasionally, the first scorer coded a response aissed by the second 
scorer or vice versa, but this did not happen frequently. Dieagreeaent at to 
Mhich critical coaponent a response belonged to was quite rare. Disagreenent 
as to level of response was soaewhat aore coaaon. Degree of agreeaent was 
calculated by treating the sequence decoaposition. need description, conaand 
finding. aDDlifcation as a four-point scale and calculating a Pearson 
correlation coefficient: the result was .92 (H-18S, p<.00i). Degree of 
accord regarding help was calculated in the saae i«y; the correlation wm .82 
(N-183, p<.001). Agreeaent as to the correctness of a response was very 
high, but this reflected the fact that vary often students' responses were 
straightforwardly correct and both scorers would so classify then. To 
provide a aore stringent teat, the scorers' agreeaent was c^txaalned on those 
responses that the scorers both classified into one or another of the 
noncorrect categorlea; there, the aecond acorer agreed with the first lt% at 
the tiae. Although thia figure ia not high, it la auch greater than the 
chance rate of 25%, aaauaing equal frequency of the categories. In general, 
then, a very satisfactory level of interscorer agreeaent ma obtained. 
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If latloit b»tti— n DTfor—aca or. Inf rvl4w and in cli^t_ 

It is mtiml to w»atr ttn&Qmt the ttwdMitt' profraMinf perforauice durirg 
tht InttrviMft «d«qtuit«ly r«fl«ct«d their ff«B«r«l procruai?* perforauace. 
Wl^t not tiM wmMal coodltloaa of intorviowlsc -> a tt ranffe interviewer, 
tlie frequent questions, even the help offered by Uie iktexvleaar — distort 
students* noraal rank order of prograaainc perfonaance? To exaaine this 
natter, indices of the students' profnaBlnc perforannce corlnf the IntervleN 
litre conpnred with their flaal course frades. for prograaainc perfomence 
during the interview, we need three iodiees: the nuaber of responses that 
were epoataneously correct, the nuaber of resolutions that had to be 
provided, aad the nuaber of the flaal probleae a atudmt atteapted. The 
figures ware draan frna appllcatioaa level reepooMe, because, as will l>e 
seea later, aubjects aade the aoet aistakea at thle level aad hr^ace their 
nuaber of correct respoaeea would be a wore individuating aeasure at that 
level than for respoaaes pooled acroas levels. 

The first row of Tabj^e 1 presents the corrt^lat5i«n coefficients 
calculated betwe«a these variables. As the Uble shtjwa, the ccrrelation 
coefficients cluster t.^Md .5?, indicating a distinct relation betaeen the 
studsnts' Interview perforaMice and their overall course pe?foraance. To be 
sure, this aagnitude of correlntion is far fron perfect, but, given such 
intervening factors aa the laterview foraat itself, studeats' response to 
Instruction subsequent to the Interview, and acddentc of perfommce both on 
the Interview and In the course, these correlations argue that at least the 
iaterview foraat did not "reshuffle" etudente' relative levels of 
perfocauce. It aqr be objected that the interview foraat perhaps led aany 
students to perfora considerably below their nosne, ahlle preeervlng relaUvi 
levels of perforaance. We have no foraal evidence on this point, but can 
renark inforaally that while soae studeots appeared nervous, others did net, 
and that, since the ;^uterview foraat itself provided direct support for 
students* problea-solving processes, a net disruptive effect appears 
unlikely. 



Insert Table 1 about here 



PerfemancJ in relatio n to attitudlnal factcra 

Aa noted earlier, the teac^ier provided attitudlnal ratinga for each atudent 
on the diaeuelona aotivatioa, eajoyaeet, aad peraiatence. Ta» ranaindar of 
Ihbie 1 displays the correlatioaa between theae ratinga and the aeaaurea c? 
perforaance during tVi interview. The correlations fall la the neighborhosd 
of .5. Ihe leoults mmfit that atndant*a effectivenesa during the interview 
related to the ttudectt*a general attitudee to prograaaing. Aa one would 
luqMct., BCi'e coaipatent prograaacra are aore aotivated, enjoy prograaaing 
aore, display moru peraiatence. 

h adding that the attitudlnal variables correlated even aore 

jti jffse grnda, in the neighbcrhood of .7. Thia aeana that they 

re: - >re atrongly to overall iourae perforaance. Thia appeara to 

give evidence of a connection between attitudea and perforaance. 

BomtVk.:'. i point ol caution, one ahould reaeaber that all ratinga were 

psovlded u» Che teacher without any aecond rater. Although the pattern of 
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Table 1 



Class Performance 


Interview Performance 




Teacher's 
Assessments 


Number of 
Spontaneously 
Correct 
Responses 


Number of 
Solutions 
Provided by 
Experimenter 


Uumber (1-8) 
of Pinal 
Problem 
Attempted 


Course Grade 


.57**** 


-.58**** 


.57**** 


Motivation 


.51** 


-.65**** 


.39* 


Enjoyment 


.38* 


-.56**** 


.25 


Persistence 


.48** 


-,52*** 


• 39* 



* p<.05, ** p<.025, *** p<.01, **** p<.005, one-tailed test, N=20, 
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results is plausible, conceivably the attitude ratings reflect a halo effect 
froa competence or vice versa or both* 

The concentration of difficulties at va rious levels 

As Mentioned earliert one aia of the present experlaent ms to detendne the 
degree of difficulty presented by the four levels of programing identified 
— deco»position» need formlationt coMand finding, and explication* This 
Mould help detwaiiie the extent to which students »uffered froa inadequacies 
of their BASIC ^'database" versus hi^er order difficulties* One relevant 
calculation exanined at each level the percentage of critical coaponents 
resolved spontaneously, rather than by my of a proapt, hint, or provide* 
The percentages for decoaposition, need foraulation, coaaand finding, and 
application respectively were 92, 78, 83, and 64* 

Me asked whether these figures differed genuinely or aerely reflected 
variatioiis In f^ Tl^ty the saae underlying distribution* A 4 x 2 chi square 
test would have been inappropriate because of iaplicit constraints on the 
aarginals resulting froa details of the scoring systea* Instead, we obtained 
a best bet underlying distribution by pooling the tallies over decoaposition, 
need foraulation, coaaand finding, and application; we performed dii square 
tests coaparing this theoretical distribution with each of the four actual 
distributions, rejecting the null hypothesis that all these percentages 
reflected the saae underlying distribution (largest chi square - 39, df«l, 
p<*001)* To underscore the significance of the percentages, 9SX of the tlae 
subjects resolved a aatter spontaneously at the decoaposition level; in 
contrast, they resolved a aatter at the application level qpontaneoosly only 
64X of the tiae* The nuabers deaonstrate that subjects encountered the aost 
difficulties at the application level* 

Another approach to coaparing levels was to consider itet percentage of 
subjects* difficulties occurred at the decoapositicn, need foraulation, 
coaaand finding, and application levels* These figures, respectively, were 
10, 25, 18, and 47X* A chi square test reflecting the null hypothesis of an 
equal distribution (25, 25, 25, 25) rejected the null hypothesis (dii square 
« 73, df«3, p<*001)* In suaaary, nearly half of subjects^ difficulties 
overall occurred at the application level* 

It is natural to ask whether the acre able subjects encountered 
relatively fewer difficulties at the application level and acre at the 
decoaposition, need foraulation, and coaaand finding levels* This aight 
occur because the aore able prograaaers would have aastered the details of 
coaaands better and so encounter fewer difficulties at the applications 
level; also, they would tackle the aore difficult probleas in the sequence, 
idiich presented aore challenges of decoaposition and need foraulation* This 
question was exaained by coaputing the percentage of difficulties at the 
application level for each subject and correlating this figure with course 
grade and with the problea nuaber (1-8) of the final problea atteapted, two 
reasonable indices of overall coapetence as indicated earlier* The 
correlations, *06 and -*31 respectively, did not approach significance, 
showing that the difficulties of the aore able subjects occurred no less 
often at the applications level than those of the less able subjects* 

Another natural question concerns whether those subjects who dealt 
successfully with at least the early phases of approaching the prograaaing 
probleas ~ decoaposition and need foraulation — approached the probleas in 
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the aaae way* After all» at leaat in principle, the prograMiing probleaa 
could be decoapoaed in aore than one nay, leading to divergent approaches* 
This question waa puraued not by a foraal imalysis but siaply by Inspecting 
over half the protocols for any sign of divergent decoapositions of the 
problems. No such cases eaerged. It seeas appropriate to conclude that 
divergent approachea were either not apparent or not attractive to the 
students* 



Effectiveness of proapts and hints 

The aoalyaia enainsd the extent to liiich proapts and hints were effective in 
helping atudeats over difficulties they encountered. Pooling the data over 
all four levela, we exaained the percentage of tiae for the entire saaple 
that a proaptt a hint, or a provide reaolved a difficulty. The figures were 
18K, and SSX respectively. In suwaary, about half the tiae a proapt or 
a hint aaaiated a atudent in reaolving a difficulty, while the reat of the 
tiae a provide proved neceaaary. The effectiveness of proapta and hints 
indicated that subjects possessed "inert knowledge" that they had not 
initially been able to nae, aince botii proapts and hinta required subjects to 
add inforwation of their own in order to reaolve the difficulty. 

We also examined whether the wore able prqgraaaers were wore responsive 
to proapts and hinta than the leaa able prograaaera. For each atudent* the 
percentage of difficulties resolved by proapts waa calculated. Also, for 
each student the percentage of difficultiea not resolved by proapts but 
resolved by hints was calculated. These figures were correlated with final 
course grnde for a aeaaure of prograwaing ability. The correlation 
cocffidaits were .28 and .25, neither of irtiich differed significantly froa 
0. It appears that the aore able prograaaers were not significantly acre 
responsive to proapts. Of course^ one aust reaeaber that the aore able 
prograaaers were reacting tc proapts given in the context of aore difficult 
probleas. 



The types of difficulties that occurred 

The data were pooled to exaaiiae the relative frequency of the four error 
types. There were 359 errors classified in all, distributed as follows: 
oaissions, Z7%; aigrations, 27%; sequence errors, S%; aistakes, 28%. We 
exMined ahether this distribution of error types varied according to idiether 
the error occurred spontaneously or in response to a proapt or hint. No 
significant difference aaong these three conditions eaerged. 

We also exaained whether the distribution of error types varied with 
level: decoapositlon, need foraulatlon, coaaand finding, and application. 
There were aubstantial differences, but they were not psychologically 
interestins, reflecting Instead the structure of the task and coding systew. 
Only one aigration occurred at the decoapoaition and need foraulatlon levels, 
becauae before reaching the coaaand finding level aubjects rarely had 
aentioned any coaaanda or coaaand structures, so there aas notiilng that could 
be said to have algrated. Sequence errors only occurred at the applications 
level, because the placeaent of coaaanda in the prograa was considered to be 
an applications-level aatter. 

The distribution of error types indicates that onr cannot interpret the 
students^ probleas with their BASIC knowledge base Just as a matter of 
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outrli^t knowledge gaps about aoM coanands* Oaiiaaiona accounted only for 
317% of the data* Niffrationa, aequence errors » and nistakea all indicated 
garbled rather than entirely Kissing knowledge about the program elenents in 
question* 



Confidence and coapetence in r elation to gender 

In lli^t of concerns about males* and fenales* attitudes and achieveMent in 
^uch traditionally sale areas as MthsMtics uA cosfMiters (cf« Badger, 1961; 
BMklns, in press), m exMined the data for contrasts between the sales and 
feaales on final grade, notivation, enJoyMnt, persistence, responsivensss to 
proBpts, responsiveness to hints, final problea atteapted, and Initial 
problea. For all but the last variable, no significant differences eac rgert . 
There mre indications that feaales found a proapt helpful a slightly lower 
percentage of tiaes than did aales, but this was at a aarginal level of 
significance, especially considering the nuaber of tests perforsed. 

A mrfced ccntrast appeared between the initial problea chosen by feaales 
with that chosen by aales. The average nuaber of the initial problea 
selected by the feaales was 1.7 and by the aales 4.2, a statistically 
significant difference (t.test, p<-003)* There was, however, a 
nonsignificant difference between feaales* and aales* final problea, the 
feaales averaging slightly lower • To test whether feaales indeed chose 
easier initial probleas without displaying a difference in ability to cover 
probleas, we conducted an analysis of variance on initial problea with final 
problea as a cover iate. The results confiraed the conclusions froa the L 
test (P»7«3, df«l, p<*015). It appears that the feaales were less confident, 
the aales aore so, in their selection of an initial problea* This points to 
an attitudinal difference that does not appear idien aales and feaales are 
compared on the other indices of ability or attitude* 



Discussion 

The analysis of the data provided information about loci of difficulty in 
three aspects of student's programming behavior: attitudes, knowledge base, 
and problem-solving strategies. 



Attitudes 

The analysis disclosed the relation one would expect between measures of 
prograaaing coapetence and attitude: More coapetcnt prograaaers, as measured 
by indices derived both froa the interview and course grade, tended to 
display aore activation, enjoyaent, and persistence as rated by their 
teacher. In addition, an interesting gender contrast appeared. While not 
differing in coapetence or other respects, feaale students evinced less 
confidence than aale students by select^.ng starting probleas that were less 
challenging. This finding is consonant with the idea that, in areas such as 
aatheaatics and coaputers, attitudinal factors aay distinguish aales froa 
feaales in our culture aore so than do natters of ability (cf . Badger, 1961: 
Hawkins, in press). 

However, the relations between attitude and coapetence allow ao siaple 
inference as to cause. It is certainly possible that students* attitudes 
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reflect their underlying coapetence in dealing with the subject Mtter of 
prograMiingt which way in turn reflect general intellectual coapetence; Idm 
(1985) reports that success in prograwwing relates considerably to IQ* On 
the other hand, wore wotivation» enJoywc.s}tt and persistmce could signal the 
presence of a wore active conception of learning that in turn fosters wore 
cowpetence (Dweck & Bewpechat, 1980; Oweck ft Licht, 1080; Zelwan, 1985). 
Farther support for this idea cowes f row our observation that sows students 
tend to be **stoppers,** disengaging frow a problew as soon as difficulties 
appear, while others tend to be ^'wovers,*' conUnuing to attack a problew that 
presents difficulties, although not necessarily effectively (Perkins, 
Hancock, Hobbs, Martin, ft and Siwwons, in press)* In general, it seews 
unlikely that the causal arrow would run only in one direction between 
attitudes and cowpetence* Host plausibly, attitudes and cowpetence would 
reinforce one another in a kind of loop* The results suggest liwit should be 
axiowatlc in education in any case: Instruction needs to take into account 
not only how students perforw but how they feel about their learning 
activities. 



Knowledge base 

It wculd be easy to consider prograwwing as a challenge principally to 
students' problew-solving abilities t since, by the weasure of any natural 
language, prograwwing languages present relatively few priwitive terws to be 
wutered* As noted at the outset, prograwwing is precision-intensive, but 
this wi^t not pose that wuch of a problew in li^t of the liwited ^'database*' 
required. However, the analysis offered awple evidence that students are 
troubled considerably by inadequacies in their knowledge base* In 
particular, students displayed wore difficulties at the application level, 
where details in the knowledge of BASIC figured wore, than at any other 
level* Osission errors and wigration errors were cowwonplace, pointing to 
problews of wissing and garbled knowledge of BASIC* 

The contewporary research on expertise leads one to expect 
knowledge-base problews in perforwance* Good problew solving in a dowain 
appears to depend upon a repertoire of schewatic problew types and aolution 
types which are specific to the dowain (e*g* Chase ft Siwon, 1973; Chi, 
Feltovich, ft Glaser, 1981; Larkin, NcDerwott, Siwon, ft Siwon, 1980; 
Schoenfeld ft Herrwann, 1982)* The iwportance of such schesnta has been 
dewonstrated for prograwwing specifically (Schneiderwan, 1978; Soloway ft 
Ehrlich, 1984)* However, it wust be ewphasized that the knowledge problews 
detected in this experiwent did not seew to be of the order of the scbewata 
detected in such studies* Rather » wany subjects fall afoul of quite 
elewentary aspects of writing individual cowwand-iines%^^ 

The analysis also dewonstrated that characterizing studttits* sboii:fall 
in knowledge as a watter of "wissing knowledge" would be siwplistic* The 
students often had relevant knowledge but failed to bring it to bear, as 
dewonstrated by the frequent success of prowpts and hints in war shelling 
students' "inert knowledge*" Inert knowledge is known to be a problew in 
other contexts of ccwposition besides prograwwing (Bereiter ft Scardswslia, 
1985)* Inert knowledge way br> considered a case of failure of transfer, 
attributable not only to the knowledge representation but to whether the 
learner windfully seeks opportunities for transfer (Belwont, Butterfield, ft 
Ferretti, 1982; Salowon ft Perkins, 1984; Perkins ft Salowon, in press)* 
Sequence errors and wigration errors also point to problews of knowledge that 
are not straightforwardly interpretable as wissing knowledge* 
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Blaewheret we have discussed a four-imy classification of students' 
knoirledBe shortfalls in prograwiing as aissing knowledge, inert knowledge , 
aispiaced knowledge > and congloaerated knowledge. Missing knowledge refers 
to knowledge that a student lacks access to. even upon prompting. Inert 
knowledge is knowledge that surfaces upon proapting or hinting or in otiier 
ways but was not initially retrieved « Misplaced knowledge weans knowledge 
that intrudes into inappropriate contexts » as with the present migration 
category. Conglomerated knowledge refers to students* occasional use of 
anomolous combinations of command elements pulled from two or more contexts. 
We introduced a new term, ""fragile knowledge,** to summarise the four and 
underscore the point that the weaknesses in students' knowledge base retires 
a diarper disracterizatioa than "missing knowledge** alone affords (Perkins & 
Martin, 1986) « 



Problem-solving strategies 

The effectiveness of prompts in helping students over difficulties argues 
that students fail to use important elementary strategies for problem 
solving. By definition, prompts were advice a student might well give 
himself or herself* Also, the prompts amount to elementary problem-solving 
strategies ~ askii« oneself what the current problem is, irtiat commands might 
helpt whether what one has Just written does what one supposes, and so on. 
The elementary daracter of these strategies deserves emphasis « We are not 
addressing the somewhat more sophisticated strategies often discussed in 
contexts of mathematical problem solving, such as constructing a visual 
representation of a problem, reducing a problem to prior problems, or 
checking a solution throu^ special cases (Polya, 1954 » 1957; Schoenfeld, 
1980; Wickelgrent 1974). To be sure, such strategies are certainly relevant 
to the context of programming; indeed, Soloway and Ehrlich (1984) have 
commented on the role of some of them. However* the present findings suggest 
that students do not help themselves enough with quite elementary 
attention-focusing strategies in dealing with programming problems. 

It must be acknowledged that the efficacy of prompts in the interview 
contexts does not in itself demonstrate that students could learn to prompt 
themselves and thereby improve their autonomous performance* However, at 
least the finding suggests that this might be possible* A contrary finding 
— that prompts had no impact on subjects* performance — would hold out 
little hope of students learning to guide their problem-solving processes 
with more art. 



Conclusion 

The results of this experiment areue that novice programmers* difficulties 
are far from monolithic. One might suppose at first that programming 
principally challenged students* problem-solving abilities. In fact, the 
results give evidence of loci of difficulty in attitudes and mastery of the 
relevant knowledge base as well as in elementary problem-solving strategies. 

The results also argue that students are troubled by some difficulties 
of a rather simple character. As noted earlier, other research has shown the 
relevance of a schematic repertoire of problem and solution types and of 
certain problem-solving heuristics to activities like programming, problem 
solving in mathematics and irtiysics, and chess play. However » our findings 
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iuggMt tbatt at least In proKraadngt ahortfalls in knowledge and strategies 
such sinpler than those usually discussed nay iapair novices* efforts 
considerably. We nay need to reassess our Intuitions about what is 
difficult: natters that one would not initially think of as difficult turn 
out to be so for the novice. It is even conceivable that an approach to 
progranning instruction which sought to inculcate nore sophisticated scheneita 
and strategies, without addressing these elenentary natters » would falter for 
lack of attention to then. Perhaps curriculun design efforts aining to help 
students with the learning and problen-solving processes involved in 
progranning should begin at quite an elenentary level. 

In particular, students nii^t be encouraged to self-pronpt tlienselves 
with the sorts of questions that focus attention on helpful aspects of 
problenatic situations. Instructicm nii^t be designed so as to help students 
to adiieve nore consolidated, less fragile knowledge of the details of the 
conputer language in question. This nay require giving students a better 
nental nodel of the conputer (DuBoulayt O'Shea, & Monk, 1981; Mayer, 1976, 
1981) as well as systenatlc franeworks for learning (Perkins, 1986). 
Finally, the findings reconnend instruction in progranning that recognizes 
tiie attitudinal problens that students nay have and tries to boost students * 
attitudes directly by whatever neans seen likely. In further research at the 
Educational Technology Center, we are investigating these avenues. 
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